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INTRODUCTION 


\ 

Finite-state  machines  provide  a  natural  means  for  repre¬ 
senting  the  logic  which  may  underlie  a  sequence  of  data  de¬ 
rived  from  a  sensed  environment  or  for  depicting  the  trans¬ 
duction  between  stimulus  and  response  of  such  an  environment. 
Such  representation  permits  expansion  of  the  logic  in  terms 
of  arbitrary  input  and  output  languages  so  long  as  these  are 
expressed  within  finite  alphabets.  Further,  the  machines  may 
be  of  arbitrary  specificity  so  long  as  they  have  only  a  finite 
number  of  states.  Thus,  no  unnatural  constraint  is  imposed, 
as  is  so  often  the  case  when  a  sequence  of  data  is  expressed 
in  terms  of  a  best  linear  fit  or  when  a  transduction  is  ex¬ 
pressed  in  terms  of  a  linear  difference  or  differential  equa¬ 
tion.  More  powerful  logical  entities  can  be  considered,  but 
these  may  be  identified  with  problems  which  reduce  their  ac- 
ceptabi 1 i ty . 

The  concept  of  evolutionary  programming  was  conceived 
as  a  means  to  find  a  most  appropriate  finite-state  machine 
for  the  purpose  of  prediction  or  modeling  in  terms  of  the 
available  data  base  and  an  arbitrary  expression  of  goal  (a 

payoff  or  error-cost  matrix).  Formulation  of  this  concept 

\ 

can  be  traced  through  various  publications.  A  review  of  the 


concepts  of  evolutionary  programming,  its  application  in  terms 
of  prediction  experiments,  data  reduction  and  analysis,  and 
control  system  design  were  described  (reference  1),  this  work 
being  supported  in  part  by  the  Office  of  Naval  Research.  This 
reference  also  indicates  application  in  terms  of  gaming  and  the 
possible  automation  of  the  scientific  method.  Further  practi¬ 
cal  and  theoretical  studies  along  these  lines  were  described 
(references  2  through  12).  These  references  concern  the  pre¬ 
diction  of  solar  flares,  the  problem  of  self-referencing,  model¬ 
ing  of  the  human  operator,  the  design  of  self-sufficient  pros- 
theses,  and  gaming  in  various  contexts. 

In  this  work  certain  problems  have  been  recognized  and 
resolved  while  others  have  been  identified  and  brought  into 
closer  perspective.  Fundamental  to  all  these  is  the  limit  of 
evolutionary  programming  in  terms  of  efficiency  and  the  poten¬ 
tial  use  of  th>s  technique  for  gaining  greater  insight  into 
aspects  of  natural  evolution. 

This  final  report  does  not  contain  previously  published 
material.  Rather,  it  includes  specific  findings  which  serve 
to  clarify  the  capability  and  limitations  of  using  automata 
and  evolutionary  programming  for  the  purpose  of  prediction 
and  control. 


DISCUSSION 


Choosing  the  size  of  a  finite-state  machine  alv/ays  pre¬ 
sents  a  problem.  Obviously  any  finite  sequence  of  symbols  can 
be  perfectly  fit  by  a  large  enough  machine,  but  it  would  most 
likely  be  a  poor  predictor,  for  the  states  would  be  exercised 
so  rarely  that  the  machine  would  not  represent  any  of  the  un¬ 
derlying  regularity  of  the  process  whereby  the  sequence  of 
symbols  was  generated.  This  problem  was  recognized  early  in 
the  development  of  evolutionary  programming  and  so  a  complex¬ 
ity  factor  was  introduced  to  inhibit  the  growth  of  machine 
size.  Initially  this  factor  was  input  data  to  the  program, 
but  to  choose  a  suitable  value  for  the  complexity  factor, 
some  a  priori  knowledge  was  required  (whether  it  is  cyclic, 
stochastic,  and  so  forth).  Various  techniques  were  developed 
so  that  the  program  could  directly  determine  machine  size  or 
indirectly  control  the  growth  of  machines  by  assigning  approp¬ 
riate  values  to  the  complexity  factor.  However,  none  of  these 
techniques  work  well  for  all  of  the  environments  which  were 
'  considered. 

The  primary  difficulty  in  this  regard  was  that  the  choice 
between  a  parent  and  offspring  had  to  be  on  the  basis  of  fit- 
score,  and  larger  machines  generally  tend  to  have  better  fit- 


scores  due  to  the  fact  that  many  of  the  state-input  pairs  are 
exercised  only  once  over  the  recall.  The  output  algorithm 
then  assigns  the  symbol  giving  smallest  error-cost  to  each 
state-input  pair  and,  thus,  assigns  the  correct  symbol  to  each 
such  singleton  state-input  pair. 

The  first  technique  used  to  eliminate  this  overemphasis 
on  fit-score  requires  that  the  program  maintain  several  classes 
of  machines,  usually  three  or  four.  One  class  consists  of  a 
one-state  machine  while  a  second  class  consists  of  machines 
having  a  number  of  states  which  exceeds  some  specified  lower 
limit  but  remains  less  than  some  upper  limit  determined  by  the 
computer  memory.  The  third  class  or  additional  classes  con¬ 
sists  of  machines  with  specified  lower  and  upper  bounds  on  the 
number  of  states.  The  lower  bounds  usually  exceed  one  and  the 
upper  bounds  are  generally  below  the  lower  limit  of  the  class 
of  large  machines. 

After  the  initial  period  during  which  the  machine  having 
the  best  fit-score  is  used  as  the  predictor,  a  machine  is  cho¬ 
sen  from  that  class  which  indicates  the  best  "action"  score 
over  the  last  k  predictions,  k  being  some  preassigned  number. 

In  general,  this  method  yielded  an  improvement  over  the  pre¬ 
vious  scheme  which  maintained  only  a  single  machine,  but  it 
increases  the  storage  problem  as  well  as  the  cost  of  comput- 
tation.  Note  that  here  several  machines  must  be  referenced 
and  the  current  machine  of  each  class  must  be  mutated  during 
each  mutation  phase. 
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Another  algorithm  was  then  devised  which  considered  the 
result  of  only  those  inputs  which  exercise  other  than  single- 
ton  state-input  pairs  in  determining  the  average  error  score 
(the  fit-score).  This  technique  as  well  as  the  earlier  algor¬ 
ithms  were  tested  against  a  first-order  Markov  process  (using 
the  single  machine  algorithm  versus  the  multiple  machine  al¬ 
gorithm  since  the  latter  always  had  a  one-state  machine  which 
would  be  the  best  finite-state  for  a  first-order  Markov  pro¬ 
cess).  The  newly  developed  technique  demonstrated  not  only 
a  better  prediction  score,  but  through  its  use,  the  size  of 
the  machines  decreased  while  machine  size  grew  in  the  older 
version . 

In  many  situations  the  plant  or  other  transducer  under 
consideration  is  actually  a  composite  of  smaller  logical  en¬ 
tities,  the  composition  being  parallel,  series,  or  a  combin¬ 
ation  of  parallel  and  series.  In  such  a  case  the  number  of 
states  in  a  single  machine  representation  of  the  transucer  is, 
in  general,  the  product  of  the  number  of  states  in  the  smaller 
machines.  For  example,  parallel  compositions  of  a  six-state 
and  a  nine-state  machine  results  in  a  54-state  machine  (al¬ 
though  the  actual  number  of  states  required  may  be  less  than 
54  since  there  may  be  some  redundancy  identifiable  and  some 
state  combinations  may  not  be  reached  from  the  given  starting 
state).  Thus,  when  a  machine  can  be  decomposed  into  smaller 
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machines,  the  total  number  of  states  which  must  be  considered 
Is  considerably  reduced.  This  also  raises  the  question  as  to 
whether  or  not  a  machine,  even  If  not  decomposable.  Is  a  sub¬ 
machine  of  a  decomposable  machine.  Thus,  theoretical  and  ex¬ 
perimental  effort  was  applied  to  the  problem  of  parallel  de¬ 
composition. 

A  program  was  written  to  evolve  a  parallel  composition 
of  two  finite-state  machine  which  would  fit  the  transduction 
of  a  larger  finite-state  machine.  In  the  initial  experiment, 
a  six-state  machine  with  eight-symbol  input  was  known  to  be 
the  parallel  composition  of  a  two-state  machine  (the  output 
symbols  of  which  were  the  high-order  bit  of  the  larger  machine' 
symbols)  and  a  three-state  machine  (the  output  symbols  of  which 
were  the  two  lower-order  bits  of  the  larger  machine's  symbols). 
After  385  mutations  the  program  successfully  evolved  the  approp 
riate  two-state  and  three-state  machines. 

Before  applying  the  program  to  arbitrary  machines,  some 
criteria  should  be  established  as  a  guide  for  determining  if 
the  program  is  evolving  machines  which,  in  parallel  composi¬ 
tion,  adequately  simulate  or  approximate  the  given  machine. 
Thus,  an  effort  was  made  to  determine  the  "best"  parallel  de¬ 
composition  of  a  given  machine  for  a  given  decomposition  of 
the  Input-output  alphabet  of  a  machine.  If  possible.  This 
would  provide  a  means  for  assessing  how  well  the  evolutionary 
program  performs.  Theoretical  aspects  of  this  problem  were 
also  considered. 
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In  the  case  of  a  machine  which  is  the  parallel  composi¬ 
tion  of  two  machines,  a  decomposition  scheme  should  produce 
these  two  machines.  This  case  was  considered  first: 

Let  =  (A.  and 

M2  =  (A,  82,  Q2»  ^2*  ^2)  be  reduced  machines  and  let 

M  =  (A,  B.  Q,  X,  6) 

be  the  parallel  composition  of  M^  and  M2  so  that  B  =  B^  x  82* 
Q  =  X  Q2,  X(a,(q^,  q2))  =  (x^(a,  q^),  X2(a,  q2))  and 

fi(a,(qi,  q2))  ~  (6i(a,  qi),62(a,  q2)). 

Let  M^  =  (A,  Bp  Q,  x,  6j)  with 

6j  (a,  (q^ ,  qg))  =  6^(a,  q^ )  and 
M^  =  (A,  B2,  Q,  X,  6^)  with 
^2  (a,  (qp  PI 2 ^  ^  ”  ^2^®*  ^2^’ 

Then  M-j  is  the  reduced  machine  of  Mj  and  M2  is  the  reduced 
machine  of  Mg. 

With  this  result  as  a  guide,  consider  a  machine  M  = 

(A,  B,  Q,  X,  6)  and  any  decomposition  of  B  into  a  direct  pro¬ 
duct  of  two  sets,  say,  B-j  and  Bg,  for  example,  B  =  B-j  x  Bg. 
Then  there  exists  unique  functions  61  and  62  with  61  mapping 
A  x  Q  into  B^  and  62  mapping  A  x  Q  into  Bg  so  that  6(a,  q)  = 
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(6^(a,  q),  62(3,  q)). 

Let  Mj  =  (A,  B^,  Q,  x,  6-j)  and 
1^2  =  (A,  B2,  Q,  X,  62) 

Then  if  under  this  decomposition  of  B,  M  is  as  before  the 
parallel  composition  of  machines  M-|  and  M2,  it  can  be  shown 
that  M-j  is  the  reduced  machine  of  M^  and  M2  is  the  reduced 
machine  of  M^. 

In  general,  if  M^  is  the  reduced  machine  of  Mj  and  M2 
is  the  reduced  machine  of  M^,  then  there  is  a  submachine  N 
of  the  parallel  composition  of  M-j  and  M2  such  that  M  is  the 
reduced  machine  of  N.  This  is  the  case  if  M  is  reduced, 
otherwise  M  and  N  have  the  same  reduced  machine. 

In  view  of  these  results,  for  a  given  decomposition  of 
the  output  alphabet,  the  reduced  machines  of  the  machines 
Mj  and  M^  constructed  as  above  from  a  given  machine  M  yield 
the  best  parallel  decomposition  of  M. 

In  the  parallel  case,  the  machines  are  essentially  inde¬ 
pendent  so  that  independent  modeling  is  possible.  This  is 
not  necessarily  true  in  the  series  composition.  If  the  in¬ 
puts  and  outputs  of  each  part  is  known,  then  the  problem  re¬ 
duces  to  that  of  modeling  individual  plants.  If  the  intermed¬ 
iate  outputs  and  hence  the  intermediate  inputs  are  not  avail¬ 
able  so  that  only  the  initial  input  and  final  output  are  known, 
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then  It  is  not  possible  to  Independently  model  the  parts. 

For  example,  consider  the  series  composition  of  plant 
P-j  followed  by  plant  P2  where  only  the  input  to  P^  and  the 

output  from  P2  are  known.  Suppose  modeling  by  the  series  com¬ 
position  of  a  finite-state  machine  followed  by  machine 

M2  is  being  attempted.  If  machine  M-j  is  kept  fixed,  then  it 

is  possible  to  evolve  the  best  machine  M2,  since  input  and 
output  to  M^  are  not  affected  by  changes  in  Mg.  However, 
when  Mg  is  fixed,  evolving  a  best  M^  presents  a  problem  in 
that  a  mutation  of  M-j  and  the  resulting  change  in  its  output 
changes  the  input  environment  to  Mg.  However,  since  the  ex¬ 
pected  output  from  Mg  is  invariant,  this  would,  in  general, 
result  in  a  degradation  of  Mg  as  a  model.  Further,  since  ex¬ 
pected  output  from  M^  is  not  known,  the  current  versions  of 
the  evolutionary  program  cannot  be  applied  since  this  know¬ 
ledge  is  required  in  their  operation.  It  was,  therefore,  felt 
that  effort  should  be  directed  toward  a  theoretical  investiga¬ 
tion  of  series  composition  to  determine,  if  possible,  effective 
algorithms  for  evolving  the  series  composition  of  finite-state 
machines.  Some  preliminary  results  are  given  here. 

Suppose  machine  M  is  the  series  composition  of  machines 
M-j  and  Mg  (composition  in  the  order  named),  where 

=  (A,  B,  Q,  Qq)  and 

^2  "  (B»  C,  P,  X2*  ^2*  ^0^’ 


Oil 


.mmmmrnm 


Then : 


M  =  (A,  C,  QxP,  X.  «. 


x(a,  (q,  p))  = 

(Ai(a, 

6(a,  (q,  p))  = 

62  (6i 

Observe  that  Q  x  P  = 

u  0 

qCQ  ' 

Q  X  P  = 

u  (q 

pCP  ' 

(Qq.  Pq);  where 

q).  \2  (6i(a,  q),  pJJ  and 
(a,  q).  p)  . 

}  X  P  ]  and  also  that 

X  {p}j 


Consider  the  first  of  these  decompositions  of  Q  x  P  and 
let  S,  -  X  P  SO  that  Q  X  P  »  US^.  Now  for  i  /  j,  S^nSj 

0  (null  or  empty  set).  Hence,  a  partition  of  Q  x  P. 

Moreover,  if  x-j  (a,  q^)  =  q^,  then 

A  (a,  S.)  =  x(a,  |q^j.  X  p) 

~  (a,  q^),  A2  ^1  (a,  q^)»  p)) 

4j}  ' 

That  is, 

x(a, 
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It  is  also  true  that  if  =  Q  x  N-  then  N  is  a  par¬ 
tition  of  Q  X  P.  However,  consider  (q^,  p^)  and  (qg,  p^)  with 
q^  ^  qg.  Both  of  those  are  in  . 

Now: 

(q^.  P,*))  =  (a,  q^),  Xg  ^6^  (a,  q^).  and 

x(a,  (qg,  p^)j  =  ^X^  (a,  qg),  Xg  ^6^  (a,  qg),  p^)j 


But,  in  general,  it  is  possible  that  5i(a,  q-j)  f  5i(a,  qg) 
and  hence  that  Xg^6^(a,  q^),  p^j  ^  Xg  ^6^(a,  qg),  p^^  . 

It  is,  therefore,  not  always  the  case  that  x(a,  T.)  CT.  for 

*  J 

some  j.  So  while  is  a  partition,  it  is  not  invariant 

under  the  x  mapping  while  the  partition  is. 

On  the  basis  of  the  partition  follows  that  for 

a  finite-state  machine  to  be  a  series  composition  of  two  ma¬ 
chines,  it  is  necessary  that  a  partition  of  the  states  exists 
which  is  invariant  under  the  x  mapping.  The  set  of  singleton 
states  forms  such  a  partition  but  this  would  lead  to  the  triv¬ 
ial  series  composition  of  the  given  finite-state  machine  with 
a  one-state  machine.  This  partition  is  excluded  as  an  allow¬ 
able  one,  for  example,  in  an  allowable  partition,  not  all  of 
the  sets  in  the  partition  can  be  singleton  sets. 

With  the  partition  |q^  x  p|  as  a  guide,  the  problem  of 
decomposing  a  given  finite-state  machine  into  a  series  compo- 
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sitlon  of  machines  is  considered.  Let  M  be  a  finite-state 
machine  where  M  -  (A,  C,  S,  x»  6.  s^)  and  suppose  S  - 

such  that  for  each  a  In  A  and  each  1  there  Is  a  j 
such  that  x(a,  S^)CSj.  (For  a  set  X,  let  |x|  be  the  cardin¬ 
ality  of  X,  that  Is,  the  number  of  elements  in  it).  If 
necessary,  relabel  the  states  so  that 


'1j 


<  J 


and  Sq  =  s^^.  Let  be  the  set  consisting  of  the  first  k 
positive  Integers,  let  B  =  A  x  and  denote  the  pair  (a^,  j) 
by  b^j.  Let 

q  =  {q,|  1  In 

be  a  set  of  k  new  symbols  and  consider  the  finite-state  machine 
with 


=  (A,  B,  Q,  x^ ,  6^ ,  q^)  where 
X-j  (a,  )  =  qj  whenever 

x(a,  S^.)CSj  and  «^(a^,  Qj.)  =  b^^.  Let 
P  -  |p,|  1  <  1  <  r} 
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be  a  set  of  r  new  symbols,  where  r  =  max|Sj|,  let  be  in  C, 
and  consider  the  finite-state  machine  Mg  with 

Mg  “  (B,  C,  P,  Xg,  «g,  p^)  where 

Xg(bij,  k^)  =  Pjjj  whenever  n  <  Sjj  and 

*jn)  = 

=  p^  otherwise  and 

Xg(bij,  p^)  *  6(a^.,  Sj^)  whenever  n  <  |sj| 

*  Cq  otherwise. 

Theorem.  M  is  equivalent  to  a  submachine  of  the  series  compo¬ 
sition  of  M-j  and  Mg.  If  jSjj  =  r  for  all  j,  M  is  equivalent 
to  the  series  composition  of  M-j  and  Mg. 

The  identification  problem  arises  whenever  it  is  required 
to  determine  a  logical  representation  for  the  transduction 
which  is  presumed  to  couple  two  sequences  of  symbols  (the  pre¬ 
sumed  input  and  output  of  a  black  box).  Evolutionary  program¬ 
ming  was  demonstrated  as  a  means  for  addressing  this  problem. 
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For  example,  the  finite-state  machines  shown  in  Figures  1  and 
2  were  driven  by  a  forcing  function,  these  input  and  output 
sequences  being  used  as  a  basis  for  modeling  by  both  the  evo¬ 
lutionary  program  and  the  human  operator.  Specifically,  Tables 
1  and  2  indicate  the  input  and  output  sequences  in  each  of  these 
experiments.  Table  3  provides  a  comparison  of  the  modeling  capa 
bility  of  the  human  operator  with  that  of  the  evolutionary  pro¬ 
gram  in  terms  of  predicting  each  next  symbol  on  the  basis  of 
the  currently  held  model. 

The  identification  problem  often  arises  within  the  context 
of  prior  information  concerning  the  equations  of  motion  or  some 
other  governing  differential  equations.  It  is  of  interest  to 
utilize  the  basic  format  of  the  equations  and  perform  an  analy¬ 
sis  of  the  available  data  base  which  will  provide  a  best  esti¬ 
mate  of  each  of  the  unknown  parameters,  thus,  completing  the 
identification  as  required.  For  example,  suppose  that  one  is 
given  a  linear  matrix  differential  equation  of  the  form 

X  =  Ax  +  Bu 

where  x  is  a  vector  of  n  state  variables,  u  is  a  vector  of  m 
control  variables,  A  is  a  time  invariant  n  x  m  matrix,  and  B 
is  a  time  invariant  n  x  m  matrix.  Find,  from  a  given  time  his¬ 
tory  of  x(t)  and  u(t)(and,  possibly,  x(t)),  the  unknown  param¬ 
eters  of  the  A  and  B  matrix. 
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MACHINE  NO 


FINITE-STATE  MACHINE  NO 
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Figure 


Tabit  1. 

INPUT  AND  OUTPUT  OF  MACHINE  I 


Input  Sequence 

1  1  2  3  1  3  2  1  1  3  1 

21133323333 
22313322323 
31223333221 
1  3  1  1  2  2  1  1  2  2  3 

3  3  1  1  3  3  1  3  3  2  1 

1  1  2  1  3  3  3  1  2  1  1 

1  3  2  2  1  1  1  3  2  1  2 

2  1  2  2  1  1  1  1  3  2  3 

13312331321 
1  1  1  2  3  1  1  2  2  1  2 

1  1  22231  23  21 

1  1  2  2  1  2  1  2  2  1  1 

2  3  2  1  1  3  1  1  1  3  2 

Output  Sequence 

23333331332 

21332333323 

3  1  3  2  3  2  1  2  1  1  1 

1  3  1  3  3  3  2  3  3  1  3 

13233133233 
13233221  3  31 

33321313133 
23333333313 

2  1  3  1  3  3  3  3  3  1  1 

21323332113 
22333333132 

3  3  1  3  1  3  2  1  1  1  1 

333132  1.  3133 
33322323333 


2  2  2  1  3  1  3  1  3  3  2 

121  12312332 

1  2  2  2  2  3  2  1  1  2  1 

321  333231  3  2 

3  2  3  1  3  1  2  3  3  3  1 

2  3  2  2  3  1  2  1  2  1  1 

1  1  1  3  2  2  3  2  2  1  3 

3223  2  312331 

12  12  2  13  12  12 

23211322312 

1  1  1  2  3  3  2  2  3  2  2 

22332332123 
3322221331  1 

2  1  3 


31213232321 
33-2  2333333  3 

21313332232 
31132333333 
11323233322 
3  3  3  1  3  2  1  1  3  2  2 

33333333333 
3  3  13  112  1113 

2  1  1  3  1  3  3  2  1  1  3 

33322333323 
133333333  3.  3 

3  13  12  112  13  3 

31231321323 
3  3  3 
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Tabit  2. 

INPUT  AND  OUTPUT  OF  MACHINE  II 


Input  Sequence 
112  3  13  2 

2  1  1  3  3  3  2 

2  2  3  1  3  3  2 

3  1  2  2  3  3  3 

13  112  2  1 

3  3  1  1  3  3  1 

112  13  3  3 

1  3  2  2  1  1  1 

2  12  2  111 

1  3  3  1  2  3  2 

1112  3  11 

1  1  2  2  2  3  1 

112  2  12  1 

2  3  3  1  1  3  1 

Output  Sequenca 

3  2  113  12 

3  2  2  1  1  1  2 

2  2  13  111 

3  2  3  1  1  1  1 

2  3  2  2  3  1  3 

2  3  2  1  1  1  3 

2  3  3  2  3  3  1 

2  3  1  2  3  2  2 

1  3  1  2  2  2  2 

3  113  2  12 

2  13  3  12  1 

2  2  2  2  1  1  3 

2  1  2  2  2  1  3 

1  1  1  3  2  3  2 


1  1  3  1  2  2  2  1 

3  3  3  3  1  2  1  1 

2  3  2  3  1  2  2  2 

3  2  2  1  3  2  1  3 

1  2  2  3  3  2  3  1 

3  3  2  1  2  3  2  2 

12  111113 

3  2  1  2  3  2  2  3 

1  3  2  3  1  2  1  2 

1  3  2  1  2  3  2  1 

2  2  12  1112 

2  3  2  1  2  2  3  3 

2  2  1  1  3  3  2  2 

1  1  3  2  2  1  3 

2  2  3  2  1  2  1  3 

11112  13  2 

2  1  2  3  2  1  2  1 

1  2  3  2  3  3  2  3 

3  1  2  3  1  2  3  2 

3  1  2  2  3  1  1  2 

3  2  2  1  3  2  2  3 

12  3  1112  1 

2  3  1  1  3  3  2  1 

2  3  1  3  3  1  1  3 

2  2  2  1  3  2  2  2 

3  113  3  111 

1  2  2  2  3  1  2  3 

1  3  3  1  2  3  1 


3  13  13  3  2 

2  3  1  2  3  3  2 

2  3  2  1  1  2  1 

3  3  2  3  1  3  2 

3  1  2  3  3  3  1 

3  12  12  11 

2  2  3  2  2  1  3 

2  3  1  2  3  3  1 

2  13  12  12 

1  3  2  2  3  1  2 

3  3  2  2  3  2  2 

2  3  3  2  1  2  3 

2  2  1  3  3  1  1 


3  2  2  2  3  1  2 

113  2  111 

2  1  2  2  2  3  2 

3  1  2  3  2  2  3 

2  2  3  1  1  1  2 

1  3  2  2  1  3  2 

1  2  1  2  3  2  3 

2  3  2  1  1  1  2 

2  2  2  2  3  2  1 

2  3  12  13  2 

1112  12  3 

2  3  1  2  2  3  1 

1  2  2  3  1  3  3 
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Fini te-State 

Finite-State 

Machine  I 

Machine  II 

Evolutionary 

Program 

99.3% 

99.3% 

Human  Operator  1 

63  % 

Human  Operator  2 

41  % 

Human  Operator  3 

40  % 

Table  3. 
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If  a  time  history  of  x(t)  is  generated  by  an  exact  solu¬ 
tion  of  the  differential  equation  above,  a  formulation  of 
n(n  m)  linear  equations  using  values  of  x(t),  x(t),  and 
u(t)  at  points  which  will  guarantee  linear  independence  be¬ 
tween  these  equations  permits  determination  of  the  exact  val¬ 
ues  of  the  n(n  +  m)  unknowns  of  A  and  B. 

If  exact  values  of  x(t)  are  not  given  yet  x(t)  and  u{t) 
for  At  sufficiently  small  are  known,  numerical  differentia¬ 
tion  of  x(t)  yields  sufficiently  small  are  known,  numerical 
differentiation  Of  x(t)  yields  sufficiently  accurate  esti¬ 
mates  o;  x(t)  so  that  again,  by  formulating  n(n  +  m)  linear 
independent  equations,  acceptable  approximate  values  for  the 
unknowns  A  and  B  can  be  found. 

If,  however,  the  time  history  of  x(t)  is  determined  by 
a  process  governed  by  the  differential  equation  noted  above, 
but  the  sequence  of  values  of  the  x(t)  is  obtained  by  measure¬ 
ments,  contaminated  by  noise,  a  solution  of  n(n  +  m)  indepen¬ 
dent  linear  equations  will  give  only  estimates  of  the  unknowns 
A  and  B.  Improved  estimates  can  be  obtained  by  solving,  in 
the  least  square  sense  (or  minimax,  which  is  computationally 
more  difficult),  a  system  with  more  than  n(n  +  m)  equations. 
This  "least  square"  solution  is  also  applicable  and  useful  in 
the  previously  mentioned  case  where  the  x(t)  have  been  obtained 
by  numerical  differentiation  of  x(t). 
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These  cases  assumed  that  the  values  of  x(t)  were  obtained 
by  a  process  which  can  be  exaccly  described  by  a  linear  differ¬ 
ential  equation  of  the  form  described  above.  However,  this 
equation  may  be  only  a  first  approximation  to  a  process  which 
is,  physically,  a  nonlinear  process.  This  is  the  case,  for 
instance,  in  the  equations  of  rotational  motion  of  an  aircraft, 
where  we  have  products  of  p,  q,  and  r  appearing  in  the  equations 
for  p,  q,  and  r.  The  problem,  thus,  consists  in  finding  two 
time  invariant  matrices,  A  and  B,  which  allow  the  approxima¬ 
tion  of  the  nonlinear  differential  equations  of  motion  by  a 
system  of  linear  differential  equations  valid  over  a  limited 
period  of  time  and  range  of  state  and  control  variables.  The 
problem  of  determining  stability  derivatives  from  flight  data 
falls  into  this  last  class. 

Looking  at  the  problem  of  system  parameter  identification 
from  the  point  of  view  of  function  optimization,  the  problem 
can  be  formulated  as  follows:  Define  a  payoff  function.  For 
instance , 


n 

z  =  E 
i  =  l 


(X 

^0 


i 


-  x^)  /w^ 


dt 


Since  x(t)  is  a  function  of  the  undetermined  parameters 
...  a^(n  +  m),  z  is  also  a  function  of  these  parameters. 
Consider  them  as  independent  variables  defining  an  n(n  +  m) 
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dimensional  space.  Here  is  the  classical  problem  of  minimiz¬ 
ing  a  function  of  n(n  +  m)  independent  variables  with  no  con¬ 
straints  . 

This  problem  can  be  attacked  in  several  ways:  A  general¬ 
ized  Newton-Raphson  method  for  finding  zeros  of  functions  of 
several  variables  may  be  used.  Several  gradient  methods  based 
on  steepest  descent  have  been  developed  in  the  past.  If  the 
number  of  independent  variables  is  relatively  large,  methods 
based  on  random  search  techniques  have  been  used  relatively 
successfully.  The  problem  of  function  minimization  with  many 
independent  variables  is  particularly  difficult  to  solve  econ¬ 
omically  when  a  comparatively  large  amount  of  computer  time  is 
required  to  evaluate  z  and  is  larger  than  the  time  required 
to  make  a  decision  about  the  necessary  changes  in  the  parameters. 
Here,  the  evolutionary  program  is  used  to  find  a  logic  which 
should  help  to  find  a  more  rapidly  converging  path  in  the  opti¬ 
mization  procedure.  The  program  used  permits  adjusting  two 
free  parameters,  these  starting  with  a  random  search  proced¬ 
ure  and  as  soon  as  enough  trials  have  been  made  using  the 
finite-state  machine  evolved  by  the  evolutionary  program  to 
decide  which  next  move  will  yield  an  improvement  in  the  value 
of  the  payoff  function. 

Situations  often  arise  in  which  it  is  desired  to  have  a 
given  system  or  plant  perform  in  a  particular  manner.  The 
problem  then  is  how  to  manipulate  the  input  parameters  so  as 
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to  achieve  the  desired  performance.  The  degree  of  difficul¬ 
ty  in  accomplishing  this  result  depends  on  many  factors,  for 
example,  the  complexity  of  the  plant  dynamics,  how  well  these 
dynamics  are  known  or  the  controllability  of  the  input  param¬ 
eter.  Consider  the  case  where  the  input  parameters  are  con¬ 
trollable  but  the  plant  dynamics  are  only  partially  known. 

A  first  step  in  controlling  such  a  plant  or  device  would  be 
to  obtain  a  workable  model  of  it.  Suppose  that  it  is  possible 
to  sample  the  inputs  and  outputs  at  fixed  time  intervals  so 
that  time  histories  of  the  input-output  pairs  are  available. 

For  the  application  of  the  evolutionary  program  a  natural 
question  is  whether  or  not  the  time  history  can  be  adequately 
modeled  by  a  finite-stated  machine.  Can  one  find  a  class  of 
plants  or  devices  so  that  sampled  time  histories  can  be  ade¬ 
quately  modeled  by  finite-state  machines?  For  such  a  class 
the  application  of  evolutionary  programming  then  becomes  feas¬ 
ible. 

Now  a  large  class  of  input-output  devices  have  their  dy¬ 
namics  representable  by  differential  equations.  Consider  those 
where  the  output  y(t)  is  related  to  the  input  x(t)  by  a  differ¬ 
ential  equation  of  the  form  y'  =  F{x,  y).  Here  F  is  assumed 
to  be  only  partially  known.  In  practice,  even  if  F  were  known, 
closed-form  solutions  do  not  exist  or  are  very  difficult  to 
derive  so  that  solutions  are  usually  obtained  by  numerical 
techniques.  Such  solutions  consider  sampled  values  at  cer- 
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tain  time  Intervals  (usually  fixed  although  sometimes  variable 
time  steps  are  used).  In  general,  the  solution  formulas  give 
the  output  value  at  the  n-th  step  as  a  function  of  the  cur¬ 
rent  Input  value  as  well  as  a  certain  number  of  the  previous 
Input  and  output  values,  for  example, 

^n  '  ’‘n-l*  *  ^n-k*  ^n-1 . ^n-k^‘ 

If  the  range  of  Input  and  output  values  are  quantized 
Into  a  finite  number  of  intervals,  then  the  solution  formula 
gives  a  function  from  a  finite  Input  alphabet  to  a  finite 
output  alphabet  which  Is  completely  determined  by  the  current 
Input  together  with  the  last  k  Input  and  output.  As  will  be 
shown  later  in  this  report,  such  a  function  can  be  represented 
by  a  finite-syate  machine  and  so  finite-state  machines  provide 
approximate  models  for  this  class  of  devices.  How  well  they 
can  approximate  such  a  device  depends,  of  course,  on  the  sam¬ 
pling  rate  of  the  Input-output  values  as  well  as  the  interval 
sizes  used  In  quantizing  the  range  of  Input  and  output  values. 

In  order  to  demonstrate  the  feasibility  of  finite-state 
machine  controlled  systems  and  to  point  out  some  of  the  diffi¬ 
culties  which  will  have  to  be  expected  In  such  systems,  a  simple 
second-order  sampled  system,  as  shown  in  Figure  3,  Is  investi¬ 
gated. 
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Quantizer 


Control  ler 


Figure  3.  Second  Order  Sampled  System 


Assume  the  system  which  has  a  finite-state  machine  as 
controller  to  work  as  follows:  After  every  T  seconds,  the 
sampler  is  closed  for  a  time  t  <<  T.  The  present  value  of 
the  output  variable  is  quantized,  and  for  our  simple  example, 
a  quantization  into  three  levels  will  be  assumed.  The  quan¬ 
tized  state  variable  x  determines  the  Input  symbol  to  the 
finite-state  machine  according  to  Table  4.  Let  the  outputs 
of  the  finite-state  machine  be  the  symbols  1,  2,  or  3  and 
let  the  following  Inputs  to  the  second-order  system  corres¬ 
pond  to  the  three  input  symbols  shown  in  Table  5.  The  finite- 
state  machine  shown  in  Figure  4  will  be  a  suitable  controller 
for  such  a  system.  In  the  form  of  a  state  transition  table, 
the  finite-state  machine  is  represented  in  Table  6.  The  dots 
in  Table  6  indicate  "don't  care"  situations.  If  properly 
synchronized,  an  input  of  2  when  the  machine  is  in  state  2 
can  never  occur.  This  shows  that  when  designing  a  controller 
in  the  form  of  a  finite-state  machine  some  thought  must  be 
given  on  how  to  properly  synchronize  the  system.  This  may 
become  one  of  the  major  design  problems  when  the  observed 
values  of  the  state  variables  which  have  to  be  controlled  are 
contaminated  by  noise.  Noisy  inputs  might  well  produce  trans- • 
itions  in  the  finite-state  machine  casuing  loss  of  synchroni¬ 
zation.  Sometimes  it  is  possible  to  design  self-synchronizing 
finite-state  machines  so  that  even  a  temporary  transition  into 
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an  improper  state  will  eventually  correct  itself.  Such  a 
self-synchronizing  finite-state  machine  may  be  more  complex 
than  a  finite-state  machine  capable  only  of  controlling  a  sys¬ 
tem  with  noise-free  measured  state  variables.  In  any  further 
analysis,  some  effort  should  concentrate  on  solving  the  control 
problem  for  noisy  state  variables. 

Figure  5  shows  the  phase  plane  plot  for  a  system  using 
the  above-described  finite-state  machine  starting  the  machine 
in  the  properly  synchronized  state.  The  phase  plane  plot  in¬ 
dicates  that  the  system  can  be  controlled  to  achieve  a  stable 
limit  cycle.  Note  that  the  system,  if  sampled  in  any  way  in¬ 
dicated  in  Figure  3,  would  be  unstable  in  general  when  using 
only  one  switching  line,  because  of  the  time-delay  between  x 
changing  its  sign  and  the  switching  of  the  forcing  function. 
Such  a  system  would  show  a  behavior  as  given  in  Figure  6. 

Even  a  system  with  two  switching  lines  where  a  zero  force 
would  be  applied  within  the  region  0  <  x  <  L  would  be  unstable 
due  to  the  time  delays  of  the  switching;  such  a  system  would 
show  a  behavior  as  displayed  in  Figure  7.  Looking  back  at 
the  system  controlled  by  the  finite-state  machine,  we  realize 
that  the  capability  of  the  finite-state  machine  to  recognize 
the  path  by  which  the  system  entered  the  state  where  0<x<L 
allows  to  apply  zero  force  when  passing  through  that  state 
in  the  direction  of  decreasing  x,  but  applying  negative  force 
when  passing  through  those  regions  with  increasing  x.  It  be- 
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comes  also  clear  that  a  controller  performing  the  same  func- 
tion  could  be  designed  if  both  x  and  x  were  available  as  inputs 

to  the  controller.  The  solution  with  the  finite-state  machine, 

• 

however,  has  the  advantage  of  not  requiring  knowledge  of  x. 

Theoretical  consideration  shows  that  within  the  accuracy 
of  the  quantizing  of  the  data  certain  finite-state  machines 
could  serve  as  models  of  transducers  whose  transfer  functions 
can  be  described  by  differential  or  difference  equations.  An 
example  of  this  is  the  modeling  by  finite-state  machines  of  a 
servo  drive  driven  by  a  random  sequence  where  the  input  and 
output  as  functions  of  time  are  known.  A  section  of  the  in¬ 
put  and  output  is  shown  in  Figure  8. 

Since  the  input  is  on-off,  a  two-level  coding  suffices. 

The  output  was  ruantized  into  80  levels.  Two  approaches  were 
considered  for  the  finite-state  machine.  One,  the  input,  would 
be  binary  and  the  past  output  would  be  reflected  in  the  states. 
This  would  require  a  large  number  of  states.  The  alternative 
would  be  to  code  the  actual  servo  input  and  the  past  output 
into  the  finite-state  machine  input,  thus,  requiring  fewer 
states.  This  latter  approach  was  selected  with  an  input  set 
of  160  symbols  and  the  output  in  80  quantized  levels. 

Two  machine  models  were  evolved  (Figure  9).  A  two-state 
machine  and  a  four-state  machine  resulted.  The  two-state  ma¬ 
chine  modeled  well  in  the  middle  part  of  the  range  where  the 
servo  output  was  fairly  linear  but  its  accuracy  fell  off  at 
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Figure  8. 
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(b)  Four-State  Machine 


Figure  9 


the  extremes  where  the  servo  output  was  non-linear.  The  four- 
state  machine  performed  as  well  on  the  linear  part  and  better 
on  the  non-linear  portion.  Both  models  tracked  the  servo  out¬ 
put  over  the  time  history  with  an  average  error  of  less  than 
five  percent. 

The  experiment  also  indicated  some  possible  difficulties 
which  must  be  considered  in  application  of  finite-state  machines 
as  models.  One  such  is  that  the  quantization  levels  and  sampling 
times  are  too  short  or  the  quantization  levels  too  gross,  the 
change  in  output  per  sample  time  may  not  be  reflected  in  the 
quantized  output  and  the  machine  becomes  locked  in  an  output 
symbol.  This  problem  occurred  on  the  servo  models  at  the  ex¬ 
tremes  where  the  slope  of  the  output  curve  was  small.  A  pos¬ 
sible  remedy  in  the  current  model  would  be  the  non-linear  quan¬ 
tization  of  the  servo  output. 

The  . possible  use  of  automata  for  controlling  unknown  or 
partially  known  plant  systems*  as  shown  in  Figure  10  were  con¬ 
sidered.  In  these  systems,  is  a  fixed  transducer  with  in¬ 
put  I  being  a  step  input  and  M2  is  a  transducer  with  transfer 
function  of  form  K/f(s).  K  represents  a  gain  and  is  a  control¬ 
lable  input,  the  task  being  to  maintain  the  output  0(t)  between 
tv'o  given  curves,  gi{t)  and  g2{t)  for  0  <  t  <  T  with  T  fixed 
and  t=0  corresponding  to  time  of  step  input. 

*The  motivation  for  the  example  is  a  control  system  of  an  air¬ 
plane  with  the  input  change  of  stick  position. 
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If  K  is  restricted  to  a  finite  set  of  values,  then,  as 
will  be  shown,  for  each  value  of  k,  the  transfer  function  in 
the  time  domain  can  be  represented  by  a  finite-state  machine, 
more  exactly,  by  a  finite  input-output  memory  machine.  A 
parallel  composition  of  these  machines  would  then  give  a 
model  of  Mg  permitting,  at  least  in  principle,  a  decision 
algorithm  to  determine  a  best  or  an  allowable  value  for  K. 

Now  such  an  approach,  particularly  if  K  can  assume  a 
large  number  of  values,  would  be  cumbersome.  Moreover,  since 
each  machine  is  actuated  by  the  same  input-output  sequence, 
few  of  the  total  number  of  states  in  the  parallel  composi¬ 
tion  are  even  exercised.  But  since  the  parallel  composition 
of  finite  input-output  memory  machines  is  also  of  this  same 
type,  an  alternative  approach  is  to  have  a  single  such  machine 
as  a  model  using  many  different  training  runs  to  evolve  it. 

It  is  important  to  consider  extended  memory  machines  and 
embedded  machines.  An  extended  memory  machine  is  a  machine, 
the  input  of  which  consists  of  the  input  symbol  sequence  to¬ 
gether  with  some  combination  of  the  last  few  input  symbols 
and  the  last  few  outputs  of  the  machine.  For  reference  pur¬ 
poses,  denote  the  combination  of  previous  input  and  output 
symbols  as  the  extended  memory. 

For  such  a  machine  with  a  given  start  :^tate  and  a  given 
initial  extended  memory,  there  is  an  equivalent  finite-state 
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machine  without  extended  memory  which  has  been  called  an  em¬ 
bedded  machine.  In  general,  different  initial  states  and  dif¬ 
ferent  initial  extended  memory  can  give  rise  to  different  em¬ 
bedded  machines. 

The  original  technique  for  exhibiting  the  existence  of 
the  embedded  machines  involved  the  finite-state  functions 
associated  with  these  machines.  While  theoretically  conven¬ 
ient  for  proving  existence  of  the  embedded  machine,  this  tech¬ 
nique  was  difficult  to  implement  and  apply  to  the  problem  of 
extracting  the  embedded  machine.  Since  in  applications  to 
modeling,  extended  memory  machines  were  often  used,  a  simpler 
scheme  for  extracting  the  embedded  machine  was  desired.  Efforts 
were  made  to  devise  one,  this  resulting  in  the  technique  des¬ 
cribed  below. 

For  simplicity  the  technique  will  first  be  described  for 
a  machine  whose  extended  memory  consists  of  the  last  output 
symbol.  Let  q^  be  a  state  of  the  extended  memory  machine  and 
let  0.  be  a  symbol  which  is  an  output  for  some  transition  into 

V 

q..  Then  the  pair  q.o.  gives  rise  to  a  possible  state  for  the 

I  *  0 

embedded  machines  as  does  any  other  such  state-output  pair. 

For  such  a  state  the  output  and  next  state  transition  assoc¬ 
iated  with  each  input  symbol  will  now  be  given. 

Let  a  be  any  input  symbol;  then  for  the  state  q.o.  the 

*  J 

output  is  that  output  when  state  q.  is  given  the  input  (o.a). 

*  0 

Suppose  it  is  Oj^.  Let  q^  be  the  next  state  reference  when 
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(o.a)  1s  the  input  to  state  q^.  Then  the  next  state  reference 
for  state  q.o.  with  input  a  is  the  state  q^o,,.  Note  that  this 
satisfies  the  requirements  for  a  possible  state  of  the  embedded 
machine.  The  actual  embedded  machine  is  the  submachine  consist¬ 
ing  of  those  states  which  can  be  reached  from  the  state  corres¬ 
ponding  to  the  next  state  and  the  initial  extended  memory. 

Consider  the  example  defined  by  Table  7  of  the  machine 
with  extended  memory  consisting  of  the  last  output  symbol. 

This  is  a  five-state  machine  with  start-state  3  and  initial 
extended  memory  of  1 .  An  input  pair  consists  of  the  ordered 
pair  of  the  last  output  and  current  input.  The  entries  in  the 
body  of  the  table  give  next  state  and  current  output  in  that 
order.  For  example,  if  the  machine  is  in  state  2  with  input 
of  13,  it  transfers  to  state  4  and  outputs  a  3.  Table  8  lists 
the  states  which  might  be  states  of  the  embedded  machine.  En¬ 
tries  in  the  table  are  interpreted  as  follows:  The  pair  listed 
in  the  next  state,  the  second  element  in  the  pair,  is  the  output. 
A  check  of  the  table  shows  that  states  11,  12,  and  31  are  indis¬ 
tinguishable  states  and  can  be  combined  into  one  state  which  is 
called  31  (since  this  is  the  start  state).  Also,  states  41,  52, 
and  53  can  never  be  reached  from  the  start  state  and  so  can  be 
dropped. 

The  resulting  machine,  which  is  the  embedded  machine,  is 
given  by  Table  9  with  (a)  being  the  next  state  reference  por¬ 
tion  and  (k)  the  output  portion.  Although  the  machine  of 
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TABLE  7. 


11 

12 

13 

21 

22 

23 

31 

32 

33 

1 

11 

23 

31 

11 

23 

11 

43 

21 

22 

2 

22 

32 

43 

22 

12 

42 

12 

42 

23 

3 

31 

23 

11 

12 

33 

21 

11 

22 

51 

4 

11 

33 

41 

13 

42 

42 

22 

33 

31 

5 

22 

32 

23 

13 

11 

42 

33 

42 

23 

Start  State  -  3 

Initial  External  Memory  -  1 
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TABLE  9 


1 

2 

3 

1 

2 

3 

13 

43 

21 

22 

3 

1 

2 

21 

22 

32 

43 

2 

2 

3 

22 

22 

31 

42 

2 

2 

2 

23 

31 

42 

23 

2 

2 

3 

31 

31 

23 

31 

1 

3 

D 

32 

31 

33 

21 

2 

3 

B 

33 

31 

22 

51 

1 

2 

1 

42 

13 

42 

42 

2 

43 

22 

33 

31 

3 

B 

51 

22 

32 

23 

2 

B 

(a)  (b) 


START  STATE  3T 
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Table  5  is  a  state  output  machine,  the  combining  of  states 
11,  12,  and  31  causes  the  resulting  machine  not  to  be  a  state 
output  machine.  Hence,  the  output  table  is  also  required. 

Consider  now  the  case  of  the  general  extended  memory 
machine  with  extended  memory  of,  say,  the  last  k  output  sym¬ 
bols  and  the  last  j  input  symbols.  A  typical  input  would 
then  be 

°1  ***  °k®l  •*• 

with  the  subscripted  portion  being  the  extended  memory.  A 
possible  state  in  the  embedded  machine  would  be  denoted  by 
the  sequence 

^i°l  • • •  °k®l  • • • 

where  q^.  is  a  state  of  the  extended  memory  machine  and 
^1  *"*  ^k^l 

is  a  sequence  of  inputs  and  outputs  whose  last  transition 
is  into  q^..  The  output  for  this  state  with  input  a  would 
be  the  output  of  state  with 

O'!  ...  0|^  ...  a^a. 
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Suppose  it  is  o.  Further,  let  be  the  next  state  reference 
when  q-j  has  input 

0]  •  •  •  ]  •  •  •  ®  j  • 

Then  the  next  state  reference  for  the  state 

q^-Oi  ...  0|^a-j  ...  a^ 

with  input  a  is 

•••  Oj^OSl^  ••• 

It  is  well  known  that  difference  equations  can  be  approxi¬ 
mately  represented  by  differential  equations.  These  in  turn  can 
be  represented  by  finite-state  machines  within  some  quantization 
error.  In  a  differential  equation  the  n-th  output  is  represen¬ 
ted  as  a  function  of  some  finite  number  of  previous  inputs  and 
outputs.  It  will  now  be  shown  that  such  a  function  can  be  rep¬ 
resented  by  a  finite-state  machine. 

Formally,  if  A  and  B  are  finite  sets  and  zA  is  the  class 

of  finite  sequences  of  elements  in  A,  then  f  is  such  a  function 

k'^  1  k 

if  f  maps  eA  into  B  and  if  there  is  a  function  g  on  A  x  B 
into  B  so  that  for  n  >  k. 
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f(a. 


aj  = 


g(an. 


‘n-1 


'n-k*  “n-1 


'n-k 


where 


=  f  (a^ . )  for  i  =  1 . n-1 . 

k-f  1  k 

(A  is  the  set  of  sequences  of  length  k+1  for  A  and  B  is 

the  set  of  sequences  of  length  k  from  B).  That  such, a  func¬ 
tion  f  can  be  realized  by  a  finite-state  machine  will  be  shown 
by  constructing  a  machine  which  will  do  so. 

Consider  the  machine  m  =  (A,  B,  S,  s^,  Xi  fi)  where  S 
consists  of  the  initial  state  s^,  a  state  for  each  sequence 

(aj,  ...  ,  a^,  bj,  ...  ,  b^ )  where  1  <  j  <  k  and 

b^.  =  f  (a^. ,  ...  ,  a^ )  for  1  <  i  f  j 

and  a  state  for  each  sequence 

(aj^,  ...  ,  a^ ,  b|^,  ...  ,  b^ ) . 

Not  all  k-tuples  of  b's  are  necessary,  only  those  which  are 
possible  values  of  f  for  the  corresponding  a's,  but  the  ex¬ 
traneous  states  pose  no  problem.  The  next  state  function  x 
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is  defined  by 


A(a,  Sq)  =  (a,  b)  where  b  =  f(a) 

x(a»(aji.-.jai»  hj,...,b.j))  =  (a»aj . a-j,  b,  bj,...,b^) 

for  1  <  j  <  k  where  b  =  f(a,  a  ^ ,  ...  ,  a-j). 

x(a,  (a|^»  ...  ]  a*!*  •••  »  bi))  —  (a^  ®|(* 

b  j  ...  f  b  2 ) 

where 

b  =  9(3*  a|^,  ...  y  a-j,  b|^,  ...  y  b-j). 

The  output  function  6  is  defined  by 
6(ay  Sq)  =  f(a) 

fi(ay  (ajf  •••  *  a-jy  b^y  ...  y  bi|))  =  f(ay  a^y  ...  y  a-j) 

for  1  <  k  <  k 

5(ay(a|^y...a‘|yb|^y...b^)}  -  9(aya|^,,..,d^yb|^y...,bi). 
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For  any  sequence  of  o  from  zA,  a  test  by  cases  on  the 

^  A 

length  of  a  shows  6(  a,  s^)  =  f(a)  where  6  is  the  extension 
of  6  from  A  to  zA,  that  is,  the  machine  M  realizes  the 
function  f.  In  general,  the  machine  M  is  not  reduced  so  that 
an  equivalent  machine  with  fewer  states  would  also  realize 
the  function.  Machines  which  realize  such  functions  are  termed 
finite  input  output  machines  with  memory  length  k.  The  above- 
constructed  machine  has  not  more  than 

1  +  (pq)  +  (pq)^  +  ...  +  (pq)*^ 

states  where  p  is  the  number  of  input  symbols  and  q  is  the 
number  of  output  symbols.  On  the  other  hand,  Massey^  has  ex¬ 
hibited  for  any  n,  machines  with  n  states  and  input-output 
memory  of  length  n(n-l)/2.  Gill  has  shown  that  for  any  n- 
state  machine,  this  is  the  maximum  memory  length. 

It  can  also  be  shown  that  the  series  or  parallel  compo¬ 
sition  of  such  finite  input-output  memory  machines  are  finite 
input-output  machines.  Since  the  evolutionary  program  mutates 


^Massey,  James  L.  IEEE  Transactions  on  Electronic  Computers. 
Vol.  EC-15,  No.  4,  August  1$66,  pages  658-65$. 

2 

Gill,  A .  Introduction  to  Theory  of  Finite-State  Machines , 
New  York,  McGraw-Hill,  Inc.,  1962. 
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the  underlying  state  structure  or  automata,  if  the  finite 
input-output  machines  have  a  well-defined  underlying  class 
of  automata,  restricting  the  mutation  to  this  subclass  should 
improve  the  efficiency  of  the  evolutionary  program  in  these 
cases  where  a  finite  input-output  machine  is  the  desired  one. 
This  was  investigated.  A  class  of  automata  was  found  which 
for  any  output  function  yields  finite  input-output  machines; 
however,  they  are  restricted  in  that  the  input  alone  deter¬ 
mines  the  output,  for  example,  for  any  input  sequence  of 
length  k  only  one  output  sequence  is  associated  with  it. 

Such  machines  would  not  suffice,  for  example,  to  model  de¬ 
vices  whose  input-output  function  satisfies  a  differential 
equation . 

On  the  other  hand,  for  any  automata  if  the  output  alpha¬ 
bet  equals  or  exceeds  the  number  of  states,  there  exists  out¬ 
put  functions  which  yield  finite  input-output  machines.  To 
see  this,  let  6  be  any  input  function  such  that  for  each  in¬ 
put  symbol  a  and  each  pair  of  distinct  states,  s^  and  Sg* 

6(a.  s-j)  f  6(a,  Sg).  Since  there  are  as  many  output  symbols 
as  states,  this  is  possible.  Hence,  for  any  given  input  sym¬ 
bol,  the  resulting  output  symbol  uniquely  determines  the  state 
and  the  machine  has  input-output  memory  of  length  two.  It, 
therefore,  appears  unlikely  that  a  suitable  subclass  exists. 
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Finite-state  machines  were  used  in  all  applications  of 
the  evolutionary  program.  More  powerful  logical  entities 
such  as  Turing  machines  were  considered,  however,  inherent 
difficulties  such  as  the  ''halting*'  problem  discouraged  their 
use.  It  was,  therefore,  appropriate  to  investigate  logical 
devices  intermediate  to  finite-state  machines  and  Turing  ma¬ 
chines.  One  candidate  in  this  regard  is  pushdown  automata. 
These  automata  should  be  more  powerful  than  finite-state  ma¬ 
chines  in  that  they  hold  some  explicit  memory.  Moreover, 
they  are  not  subject  to  the  halting  problem.  In  view  of  the 
fact  that  mutation  within  the  evolutionary  program  is  applied 
to  the  overlying  finite-state  automata  (the  output  symbols 
being  deterministically  generated  so  as  to  provide  the  best 
fit  of  the  data  for  the  given  automata),  it  appears  particu¬ 
larly  appropriate  to  consider  pushdown  automata. 

A  pushdown  automata  M  is  a  6-tuple,  (A,  C,  Q,  6,  z^,  q^), 
where  A  is  the  input  language,  C  is  the  pushdown  alphabet, 

0  is  the  set  of  states,  is  a  special  pushdown  symbol, 
is  the  initial  state,  and  6  is  a  mapping  of  AXCXQ  into  C*xQ. 
(C*  is  the  set  of  all  finite  words  which  can  be  made  from 
the  alphabet  C  including  the  empty  word  e. )  Intuitively, 
with  each  triple  consisting  of  an  input  symbol,  a  pushdown 
symbol,  and  a  state,  6  associates  a  pair  consisting  of  a 
word  made  up  of  symbols  from  the  pushdown  alphabet  and  a 
state  (the  next  state).  Translating  this  into  machine  opera- 
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tion,  suppose  the  automata  has  the  word  w  In  the  pushdown 
store  where  w  =  xc  with  x  a  word  in  C*  and  c  in  C,  is  in 
state  q-j  and  is  given  the  input  symbol  a.  If  6(a,  c,  q-j)  = 
(y*  ^2^*  automata  goes  into  state  ^2  word 

xy  is  now  in  the  pushdown  store.  Note  that  y  may  be  the 
empty  word  in  which  case  the  word  in  the  pushdown  store 
would  be  X,  for  example,  the  symbol  c  is  "erased."  Under 
this  convention,  it  is  the  last  symbol  in  the  pushdown  store 
that  is  read  so  that  the  pushdown  store  is  essentially  a 
last-in,  first-out  device.  The  symbol  is  the  initial 
symbol  in  the  pushdown  store  and  to  prevent  emptying  of 
the  store  and  the  resulting  premature  halting  of  the  autom¬ 
ata,  erasing  of  is  no  allowed,  that  is,  it  is  never 
the  case  that  6(a,  z^,  q)  =  (e,  q'). 

The  question  remains  as  to  whether  or  not  pushdown 
automata  are  more  powerful  than  finite-state  machines  when 
used  as  predictors.  Only  a  partial  answer  to  this  question 
can  be  offered.  Note  that  a  finite-state  machine,  when  used 
as  a  predictor  (that  is,  the  last  output  symbol  is  used  as 
the  next  input  symbol),  generates  output  sequences  which  are 
eventually  periodic.  It  is  of  interest  to  determine  whether 
or  not  pushdown  automata,  when  used  as  predictors,  will  gen¬ 
erate  only  sequences  that  are  eventually  periodic. 

The  action  of  a  pushdown  store  automaton  is  determined 
by  a  triple  consisting  of  the  present  state  of  the  automaton. 
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the  input  and  the  top  symbol  in  the  pushdown  store.  The 
automaton  then  transfers  to  a  next  state  (possibly  the  same), 
outputs  a  symbol,  and  changes  the  pushdown  store.  Three  dif¬ 
ferent  types  of  changes  in  the  pushdown  store  can  occur.  The 
top  symbol  may  be  erased  and  the  next  symbol  from  the  top  be¬ 
comes  the  top  symbol,  the  top  symbol  may  be  replaced  by  a 
single  symbol,  or  the  top  symbol  may  be  replaced  by  a  word 
of  length  greater  than  one.  Call  a  triple  which  results  in 
an  erasure  a  decreasing  triple  and  a  triple  which  results  in 
the  top  symbol  being  replaced  by  a  word  of  more  than  one  sym¬ 
bol  an  increasing  triple.  . 

If  a  pushdown  store  automata  has  no  increasing  triples, 
then  only  the  initial  pushdown' symbol  ever  occurs  in  the  push 
down  store,  and  the  automata  is  effectively  a  finite-state 

machine.  So  there  is  no  loss  in  generality  in  assuming  the 

/ 

automata  has  at  least  one  increasing  triple.  A  sequence  of 
successive  triples  for  an  automaton  is  called  an  increasing 
sequence  if  it  contains  at  least  one  increasing  triple  but 
no  decreasing  triples.  Similarly,  a  sequence  is  a  decreas¬ 
ing  sequence  if  it  contains  at  least  one  decreasing  triple 
but  no  increasing  triples. 

Lemma  1.  When  used  as  a  predictor,  a  pushdown  store 
automata  has  increasing  sequences  of  bounded  length  or  it 
has  an  increasing  sequence  of  infinite  length.  If  it  enters 
an  increasing  sequence  of  infinite  length,  then  the  output 
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is  eventually  periodic. 

There  are  only  a  finite  number  of  states,  of  input  symbols, 
and  of  pushdown  symbols.  There  are,  therefore,  only  a  finite 
number  of  increasing  triples,  say,  k  of  them.  Similarly,  there 
are  only  a  finite  number,  m,  of  triples  which  are  neither  in¬ 
creasing  nor  decreasing.  Suppose  there  exists  an  increasing 
triple  of  length  at  least  k  +  m  +  1,  then  either  an  increasing 
triple  or  one  which  is  neither  increasing  nor  decreasing  must 
be  repeat;‘d.  Since  in  a  sequence  of  triples  none  of  which  is 
decreasing,  only  the  top  pushdown  symbols  are  ever  used,  the 
sequence  of  triples  between  repetitions  of  a  triple  is  then  re¬ 
peated.  The  resulting  increasing  sequence  is  then  infinite  and 
the  output  cyclic. 

Lemma  2.  When  used  as  a  predictor,  if  a  pushdown  store 
automata  has  decreasing  sequences  of  triples  of  bounded  length, 
the  output  is  eventually  cyclic. 

If  there  is  an  infinite  increasing  sequence  which  can  be 
entered,  then  Lemma  2  follows  from  Lemma  1.  Consider  then  the 
case  when  all  increasing  sequences  are  bounded.  Suppose  the 
bound  on  the  length  of  the  decreasing  sequences  is  k,  then  if 
the  automata  is  actuated  by  a  decreasing  triple  the  ensuing 
actions  of  the  automata  until  an  increasing  triple  is  encoun¬ 
tered  are  determined  by  the  current  triple  and  the  top  k  sym¬ 
bols  in  the  pushdown  store  or  by  the  word  in  the  pushdown  store 
if  it  contains  less  than  k  symbols.  Since  there  are  only  a 
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finite  number  of  decreasing  triples  and  a  finite  number  of  k- 
tuples  of  pushdown  symbols,  the  automata  eventually  must  have 
the  current  triple  and  the  top  k  symbols  on  the  pushdown  store 
a  repetition  of  a  previous  one  and  so  a  cyclic  output  from  that 
point  on. 

As  yet  unresolved  is  the  case  of  an  automata  which  when 
used  as  a  predictor  has  increasing  sequences  of  bounded  length 
but  decreasing  sequences  of  unbounded  length.  If  such  a  machine 
could  exist  then  the  possibility  of  other  than  eventually  cyc¬ 
lic  output  exists.  Efforts  to  devise  such  a  machine  or  to  prove 
it  cannot  exist  have  as  yet  been  unsuccessful. 

Thus,  pushdown  automata  appear  to  be  more  powerful  than 
finite-state  machines  as  predictors  even  though  the  output  of 
such  a  machine  when  used  as  a  precictor  is  eventually  periodic. 
Specifically,  pushdown  automata  have  the  advantage  that,  with 
a  given  number  of  states,  longer  cycles  can  be  generated  than 
by  finite-state  machines. 

The  next  logical  device  to  be  considered  is  the  stack 
automata.  Again,  as  in  the  case  of  both  the  finite-state  ma¬ 
chines  and  pushdown  automata,  only  the  deterministic  form  of 
the  automata  will  be  considered.  The  stact  automata  has  the 
same  basic  structure  as  the  pushdown  automata  with  the  addi¬ 
tional  ability  to  read  any  symbol  in  the  stack,  not  just  the 
topmost  symbol.  A  generalization  of  the  stack  automata  has 
also  been  introduced  in  which  the  input  tape  can  be  run  both 
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forward  and  backward  or  remain  stationary  and  is  called  a  two- 
way  stack  automata.  The  form  considered  here  Is  the  one-way 
stack  automata  in  which  only  forward  movement  or  no  movement 
is  permitted.  The  two-way  stack  is  prone  to  the  "halting  prob 
lem."  Since  the  stack  pointer  or  reader  moves  one  symbol  at  a 
time,  the  stationarity  of  the  input  tape  is  required  so  as  to 
allow  the  machine  the  potential  of  reading  any  symbol  on  the 
pushdown  stack. 

To  show  that  a  stack  automata  is  actually  more  powerful 
than  the  pushdown  automata,  consider  the  set  |wcw|  where  w  is 
any  word  in  |a,  b[*  and  c  is  a  symbol  different  from  a  and  b}. 

3 

It  is  known  that  this  set  of  words  is  not  a  context-free 
language  and,  hence,  is  not  acceptable  by  a  pushdown  automata. 
However,  it  is  acceptable  by  the  stack  automata  which  is  now 
descri bed. 

The  stack  automata  consists  of  four  states,  (1,  2,  3,  and 
4),  with  1  as  the  initial  state  and  {3}  as  the  set  of  terminal 
states,  for  example,  3  is  the  only  terminal  state.  In  the 
initial  state  it  duplicates  the  input  symbol  on  the  pushdown 
stack,  moves  to  the  next  stack  position  and  next  input  posi¬ 
tion,  and  remains  in  state  1  unless  the  symbol  is  a  c  or  a 


nsburg,  Seymour.  The  Mathematical  Theory  of  Context-Free 
Languages ,  McGraw-Hill,  Inc.,  New  Vork,  1966. 
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blank  which  indicates  end  of  input.  Hence,  any  word  w  without 
a  c  in  it  is  duplicated  in  the  pushdown  store.  If  a  blank  is 
input,  the  machine  stays  in  state  1  and  stops.  When  a  c  is 
the  input  in  state  1,  the  machine  moves  the  stack  pointer  back 
to  the  first  stack  symbol,  moves  the  input  ahead  one,  and  goes 
to  state  2.  In  state  2,  as  long  as  the  input  tape  is  not  blank 
and  the  input  tape  and  stack  agree,  it  moves  the  stack  and  in¬ 
put  ahead  one  position  and  remains  in  state  2.  If  the  symbols 

disagree,  it  goes  to  state  4  where  it  stays  for  any  input  and 

stack  symbol  stopping  the  machine  on  a  blank  input.  If  the  ma¬ 
chine  is  still  in  state  2  when  end  of  tape  is  reached,  for  ex¬ 
ample,  a  blank  is  input,  and  if  stack  symbol  is  blank,  it  moves 

to  state  3  and  machine  stops,  otherwise,  it  stays  in  state  2  and 
stops.  Hence,  the  maichine  reaches  state  3  and  stops  there  only 
if  the  input  is  of  form  wcw  with  w  in  |a,  b[*. 

Intuitively,  the  pushdown  store  cannot  recognize  a  word  of 
form  wcw  since,  although  w  can  be  stored,  the  machine  reads  it 
backward  and  so,  except  for  polindromes,  cannot  correctly  com¬ 
pare  the  word  after  the  c  with  the  word  in  front.  Note  that  a 
pushdown  store  can  accept  jwcw*^  j  where  w*^  is  the  word  w  reversed} 

D 

since  reading  w  backwards  is  equivalent  to  reading  w  forward. 

The  usual  formulation  for  the  stack  automata  requires  that 
the  input  alphabet  A  contain  a  special  symbol  3*  the  terminal 
symbol.  A  word  w  from  |A  -  jg}}  *,  for  example,  w  contains  no 
B's,  is  accepted  by  M  if  whenever  the  word  wg  is  input  with  M 
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In  state  s^,  then  when  symbol  ^  Is  read  the  machine  enters 
one  of  the  terminal  states.  The  symbol  e  is  not  necessary, 
but  simplifies  the  formulation.  For  a  two-way  stack  automata 
an  initial  input  symbol  is  needed  to  prevent  running  off  the 
beginning  of  the  tape. 

When  used  as  a  transducer  or  predictor,  the  terminal 
symbol  is  unnecessary  since  in  this  application  there  is  no 
concern  for  a  last  input  symbol.  Also,  the  set  of  terminal 
states  is  replaced  by  an  output  language  C  and  the  result  of 
6  or  6^  is  now  a  quadruple  rather  than  a  triple.  The  first 
three  coordinates  are  as  before  (for  example,  next  state, 
movement  of  input  tape,  movement  of  stack  pointer,  and  any 
writing  or  erasing  of  stack  symbols)  with  the  fourth  coordin¬ 
ate  the  output  symbol.  One  of  the  possible  output  symbols  is 
the  blank  or  equivalently  no  output  so  as  to  allow  the  stack 
pointer  to  move  more  than  one  space  before  an  output  is  re¬ 
quired. 

Previous  analysis  indicated  that  when  used  as  a  predic¬ 
tor  the  pushdown  automata  would  output  only  cyclic  sequences 
and  so  does  not  improve  on  the  finite-state  machine  as  a  pre¬ 
dictor.  A  stack  automaton  will  now  be  given  which  outputs  as 
a  predictor  the  sequence  0100110...,  for  example,  al¬ 
ternate  sequences  of  O's  and  I's  of  increasing  length,  which 
is  certainly  not  cyclic. 

Let  M  be  the  stack  automaton  with 
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^  °  l®o’  *1 ’  *2’  ®3l 


A  =  fO.  1 t 

B  =  10,  1, 

0  '  |o.  1.  At 

and  6  and  defined  as  follcws: 


i(So. 

■  *  "o^ 

=  (^0 

,  N.  R 

.  a) 

«(s„.  ■ 

'  .  1)  = 

R.  R, 

1) 

sASg, 

•  ,  a) 

=  (S^: 

.  N,  N 

1. 

6(S^,  • 

.  1)  = 

(s^ , 

N.  2, 

a) 

6(S^,  • 

*  "o^ 

=  (Sg. 

.  N,  R. 

,  a) 

6(Sp.  • 

.  1)  = 

(Sg, 

R.  R. 

0) 

The  •  Indicates  either  a  0  or  a  1  can  be  entered. 

* 

N  1  indicates  no  move  and  writing  a  1  on  the  blank  space 
being  read. 
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fiA(s2»  •  ».a)  ~  (s2»  Lj  A) 

*  *  1)  “  (^3* 

^(Sjl  I  ~  ^^0* 

The  remaining  triples  on  which  6  and  should  be  defined 
cannot  occur  and  so  are  not  included. 

A  description  of  the  machine's  actions  follows.  At  the 
start  it  is  in  state  s^  with  the  0  or  1  as  input  symbol  and 
stack  symbol  z^.  It  stays  in  state  s^^  with  same  input,  but 
moves  stack  symbol  ahead  to  next  space  which,  at  this  time, 
is  blank.  It  then  prints  a  1  in  this  blank  space,  does  not 
move  input  tape  or  output  a  symbol,  and  moves  to  state  Sy 
Other  wise,  when  it  enters  s^,  the  stack  pointer  is  on  the 
bottom  1,  the  machine  then  moves  both  the  input  tape  and 
stack  pointers  right  and  outputs  a  1  for  each  1  on  the  stack. 
When  the  pointer  leaves  the  word  on  the  stack  to  the  first 
blank,  the  machine  prints  a  1  there,  does  not  move  the  input 
tape,  does  not  make  an  output,  and  goes  to  state  Sy  In  state 
S'!  there  is  no  input  tape  movement  nor  output,  the  machine 
moves  the  stack  pointer  back  until  it  reaches  staying  in 
state  s-j.  When  the  pointer  reaches  z^,  it  is  returned  to  the 
bottom  1  on  the  stack  and  the  machine  goes  to  state  S2  without 
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moving  the  input  tape  or  outputing  a  symbol.  In  $2  the  ma¬ 
chine's  actions  are  as  in  s^  except  the  output  is  a  0  rather 
than  a  1  and  when  the  stack  pointer  reaches  the  first  blank 

space,  the  machine  moves  it  back  to  the  topmost  1  on  the 

stack  and  goes  to  state  s  .  State  s^  is  a  duplicate  of  state 
S'!  except  when  the  stack  pointer  reaches  the  symbol  z^,  the 
machine  moves  to  state  s„  rather  than  state  So.  This  com- 
pletes  a  cycle  and  the  machine  goes  through  the  cycle  over 

and  over  except  during  each  cycle  the  number  of  I's  on  the 

stack  is  increased  by  one  and  consequently  so  is  the  length 
of  the  output  sequences  of  zeroes  and  ones.  For  convenience, 
this  machine  was  constructed  so  as  to  be  independent  of  the 
input.  By  making  the  output  depend  on  the  input,  different 
output  sequences  would  result  depending  on  the  initial  input 
symbol . 
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CONCLUSION 


Finite-state  machines  provide  a  natural  basis  for  the  pre¬ 
diction  of  time  series  and  the  modeling  of  an  unknown  transducer. 
Evolutionary  programming  offers  a  means  for  finding  suitable  fi¬ 
nite-state  machines  with  respect  to  a  given  goal  (payoff  or 
error-cost  matrix)  so  long  as  the  cost  of  required  data  proces¬ 
sing  does  not  become  excessive.  Tracing  the  average  cost  of 
computation  in  the  recent  past  makes  it  reasonable  to  expect 
much  greater  efficiency  in  coming  years.  The  advent  of  intrin¬ 
sic  parallel  computers  would  open  new  prospects  for  the  realiza¬ 
tion  of  prediction  and  modeling  through  the  continual  search  for 
automata  which  are  most  appropriate  to  the  situation  at  hand. 

In  fact,  such  mechanisms  might  permit  improving  population  prop¬ 
erty  constraints  on  the  inheritance  of  the  evolving  organisms. 

Various  possibilities  for  governing  the  nature  and  amount 
of  mutation  "noise"  have  been  considered.  However,  here  there 
is  an  essential  trade-off  between  efficiency  and  securi ty . . . the 
more  tightly  linked  the  mutation  noise  to  the  successful  kinds 
of  mutations  which  have  occured  in  the  past,  the  less  the  like¬ 
lihood  of  finding  radical  departures  in  terms  of  the  logic  of 
prediction  and  control.  The  greater  the  efficiency  in  the 
search  for  each  new  logic  within  a  restricted  class  of  poten- 


OGl 


tial  environments,  the  less  the  versatility  of  the  program  and 
the  more  predictable  becomes  its  behavior.  In  the  face  of  a 
competitive  environment,  it  may  well  prove  worthwhile  to  be 
less  efficient  and  most  versatile,  even  at  far  greater  compu¬ 
tation  cost. 

Evolution,  indeed,  the  scientific  method,  occurs  at  vari¬ 
ous  levels  of  abstraction.  It  is  clearly  worthwhile  to  restrict 
attention  to  those  models  which  have,  in  the  past,  been  found 
most  worthwhile  and  to  model  these  as  a  basis  for  further  pre¬ 
diction  and  control.  In  fact,  models  of  models  can  be  used  at 
all  levels,  provided  the  sample  size  is  adequate.  Difficulty 
is  encountered,  however,  in  that  the  alphabet  of  possible  sym¬ 
bols  is  much  greater  at  each  higher  level.  Wise  judgment  as  to 
the  use  of  a  reduced  alphabet  of  descriptors  may  prove  worthwhile 
or  the  very  essence  of  the  higher-level  modeling  may  have  been 
lost.  Such  selection  of  an  appropriate  alphabet  reduction  forms 

a  meta-problem  which  can  only  be  dealt  with  within  a  particular 

( 

well-defined  frame  of  reference.  Such  evolutionary  programming 
remains  to  be  explored. 

The  immediate  problem  of  this  investigation  has  been  to 
devise  new  means  for  solving  immediate  real  world  problems  of 
Naval  interest.  Thus,  with  this  intent,  attention  has  been  fo¬ 
cused  upon  developing  an  improved  logical  capability  with  re¬ 
spect  to  a  wide  class  of  practical  problems.  Hopefully,  this 
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technique  can  also  be  used  to  Investigate  the  sensitivity  of 
prediction  and  control  with  respect  to  various  goals.  Hope¬ 
fully,  it  may  require  explicit  consideration  of  the  problem  of 
choosing  meaningful  descriptors  and  goal  formulation. 
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